System and method for simultaneously viewing, coordinating, manipulating and interpreting three-dimensional and two-dimensional digital images of structures for providing true scale measurements and permitting rapid emergency information distribution

ABSTRACT

The present invention provides a true-scale, coordinate-matched, linked in real-time, dual three-dimensional/two-dimensional visual display/viewer. The display simultaneously shows a 3D digital image and an associated 2D digital image of a selected drawing. The display of the present invention allows a user to visualize an asset&#39;s location, surrounding environment and hazards and true scale structural details for interior or external structural scenes. Using the display and associated tools, the user can obtain real-time information of an environment, true-scale measurement, plan ingress/egress paths, shortest paths between points and the number of doorways in a structure and track objects within the displayed environment. The intelligence gained using the tools and 3D/2D display may be used and further manipulated by a single user or may be distributed to other users.

PRIORITY AND RELATED APPLICATIONS

The present invention is a continuation application of U.S. patentapplication Ser. No.: 12/610,441, filed Nov. 2, 2009, entitled “SystemAnd Method For Simultaneously Viewing, Coordinating, Manipulating AndInterpreting Three-Dimensional And Two-Dimensional Digital Images OfStructures For Providing True Scale Measurements And Permitting RapidEmergency Information Distribution”; which is a continuation-in-part ofU.S. patent application Ser. No. 11/068,268, filed Feb. 28, 2005,entitled “System And Method For Rapid Emergency InformationDistribution” now U.S. Pat. No. 7,634,156; which is acontinuation-in-part of U.S. patent application Ser. No. 10/629,347,filed Jul. 28, 2003, entitled “Systems And Methods For Providing TrueScale Measurements For Digitized Drawings” now U.S. Pat. No. 7,672,009;which claims the benefit of U.S. Provisional Patent Application Ser. No.60/398,927, filed Jul. 27, 2002, entitled “Systems and Methods forViewing and Modifying Digitized Drawings”; and further claims thebenefit of U.S. Provisional Patent Application Ser. No. 60/547,790,filed Feb. 27, 2004, the contents of each are hereby incorporated byreference as if set fully herein. PCT application PCT/US2009/62918entitled “System and Method Employing Three-Dimensional andTwo-Dimensional Digital Images” was filed on November 2, 2009 concurrentwith U.S. patent application Ser. No. 12/610,441.

FIELD OF THE INVENTION

The present invention relates generally to digital images, and moreparticularly, to the viewing of digital images.

BACKGROUND OF THE INVENTION

The scanning of paper documentation into digital images is well known.Some of the advantages of digital or electronic documents over paperdocuments include reduced storage space, immediate and simple copying,quick retrieval, easy sharing through electronic transfer (e.g.,e-mail), persistent and non-volatile nature of a digital format, and theconservation of natural resources such as trees. While a completelydigital office is not a reality for most businesses, it is rare to finda business that doesn't rely heavily on digital documents in theordinary course of its business.

For example, property owners, land developers, architects, and documentmanagement professionals scan active and historical documents relatingto properties, such as building blueprints, floor plans, and riserdiagrams, to save space and enable more efficient copying anddistribution of the documents. However, once a drawing is scanned, thescale information on the drawing is not computer recognizable when thedigital version of the paper drawing is viewed on a monitor or displaydevice. In particular, the digital image of the drawing is typicallycaptured as a digital image having a certain pixel by pixel dimensionwith no direct or easy means to establish a relationship to the scaleinformation contained on the original drawing. Thus, when the image isviewed using a monitor or display, it is virtually impossible for theuser to obtain true measurement information from the rendered imagebecause the scale of the paper drawing, for instance, one inch equalsthree feet, is not valid for the rendered image on the monitor ordisplay.

Traditionally when paper plans are scanned and digitized for electronicstorage, the images original physical size, and therefore thecorresponding usefulness of the image scale, of a particular document isno longer a concrete attribute of the image. For example, if a paperversion of an infrastructure plan is thirty inches in height and fortyinches in width and then scanned, a computer user of that scannedelectronic image would see the document as a different physical sizewhen using different monitors depending on the size of the displaydevice and its own pixel resolution. Thus, the scale that appears on thedocument (e.g., one eighth inch equals one foot, etc.) will be incorrectwhen an electronic depiction of the document is displayed on a computermonitor. This is because the original physical size of a paper image hasno direct correlation to the pixel dimensions of a computer monitor. Asa result, a 20 inch wide monitor can only display an image as twentyinches wide if viewing the whole image and a twenty-five inch widemonitor can only display an image as twenty-five inches wide if viewingthe whole image. Also, neither monitor would be able to display thewhole image as it originally appeared, that is, as a forty inch wideimage. The user has no way to know what the original physical size ofthe paper drawing was, yet the scale ratio of the image listed on theplan is directly tied to the physical size of the original paperdocument. So if a computer user viewing the scanned infrastructure planon a twenty-five inch monitor tried to take a physical measurement ofthe image on the computer monitor using that data with the image scaleto manually compute a true scale measurement the result would be a wrongmeasurement value. Furthermore zooming the image so that only portionsof the original image appear on the computer monitor also distorts thephysical size of the image making any physical measurement of an imageor image element not useful when combined with scale to calculate a truescale dimension measurement. In essence, once a paper drawing isscanned, the scale information on that drawing is no longer valid andaccurate when a digital version of the paper drawing is viewed on amonitor or display device.

Accordingly, some of the utility inherent in paper documents is lostwhen the documents are digitized. This lost utility is particularlyproblematic when it is desirable to determine the measurements of aroom, the length of a wall, or the square footage of a section of afloor, which is often the main reason for viewing the drawings. Inaddition, when annotating the digital drawing, it is often desirable toannotate where the graphic annotations retain a true scale ratio to therendered subject matter represented on the digital image.

Thus, there exists an unsatisfied need in the industry for a means toview, and distribute a digital drawing with the ability to determinetrue and precise dimension information which accurately describes therendered subject matter.

Also, it is known that event information regarding buildings can bedisplayed with the digital drawing of buildings. For instance, it isknown that buildings can be provided with various alarm systems. U.S.patent application Ser. No. 10/434,390 discloses a method of displayingevent information from a building system where the event is a non-normalcondition generated within a building system. Information regarding thebuilding is displayed on a display portion. The displayed information isselectable and changeable by a user. An alarm graphic can also bedisplayed which relates to a non-normal condition in a building. A usermay elect to show a floor plan, which discloses the status of firesystem alarm generating devices. However, while this graphic may bedisplayed, the user is unaware of the accurate to-scale spatialrelationships that exist between people in the building, the non-normalcondition, and the building's structural characteristics.

A responder assets management system (RAMS) is disclosed in U.S. patentapplication Ser. No. 10/038,572. The disclosed system utilizesinformation available to responders including emergency responsepersonnel including local weather, national weather, and links to otherinformation. The system also provides virtual walkthrough capability ofa building or facility. However, while providing this virtualwalkthrough, there is no ability for the user to scale and zoom todetermine exact spatial relationships.

Finally, U.S. patent application Ser. No. 10/177,577 discloses a systemand method for detecting monitoring and evaluating hazardous situationsin a structure. Sensors having two-way communication capability arestrategically located in a structure or in a matrix of structures. Theseunits are high-level multi-functional detectors that communicate with abase computer. However, as with the other systems discussed above, thereis no spatial relationship provided for users so that they can determinetheir exact relationship to the hazardous situations within a structure.

Spatial relationship is further indeterminable in the prior art due tothe type of displays, viewers, or graphic view ports, used to view thegraphically represented floor plans or drawings. Two traditional typesof displays used in the prior art are either 2D displays or 3D displays.Though each display provides users individual benefits, these benefitsare limited. For instance, a 2D display can be used by a user to plot aspace with respect to the entire building or structure, however the 2Ddisplay cannot describe the complete geometry nor visual qualities ofthe interior of a room or passageway of a structure. In such cases, whena user is using the 2D display of a floor to plot entry or exit routesin a structure, details regarding the architecture and geometry of aparticular route cannot be comprehensively determined as they could bein a true scale 3D animation or true scale 3D virtual representation ofthe space. Also, use of only 2D displays does not permit routeadjustments to be made for architectural and hazardous elements visuallyidentified in the building that arise along a navigated path. Forexample a 2D floor plan may indicate that a particular passage way iswide enough for a particular piece of equipment, however the actualheight and architectural geometry of the passage way in all dimensionscannot be represented in the two dimensional representation. As a resultemergency teams or other building system workers are being presentedwith incomplete data that can directly cause bad or hazardous decisionswhen using only the 2D floor plan as decision support tool.

The benefits of viewing a floor plan when using only a 3D display alsois compromised as the user merely views the interior of a structurewithout being able to quickly identify the wall construction andembedded electrical, natural gas, and plumbing details. Additionally,users using 3D displays can only observe the spatial relationship forobjects in a room that are directly in their cone of vision and areunaware of potentially hazardous/important adjoining room/areacharacteristics, including, but not limited to blocked passages,location of hazardous materials, alarm status and other critical itemsof importance.

The prior art provides for visualization of graphics either in 2D or in3D in isolation, however a display is needed that provides improvedviewing capabilities to take advantage of a novel visual datarelationship created by the invention. Such a display would provide thecumulative advantage of both the 3D and 2D displays. Thus a display isneeded that would create a synchronized true scale visual relationshipbetween two related and connected but independent data perspectives in away unseen in previous technologies. A display is needed that can form asymbiotic data visualization between true scale 3D and 2D displays,which is not realized when such displays are viewed independently, evenwhen viewed in succession. Such a display would permit simultaneousdisplay of a route in 3D and 2D with concurrent access to critical,measurable, spatial and relationship data via a true scalecoordinate-linked display. It is also desired that such a display wouldproduce an accurate, true scale measurement of the route.

BRIEF SUMMARY OF THE INVENTION

The present invention provides a true scale, coordinate matched, linkedin real time, dual three-dimensional/two-dimensional visual display(viewer). The combined, simultaneous and real-time visual display of thepresent invention can be used to effectively assess risks, define safeand kill zones, visualize critical assets, alarms and sensor, hazardsand true scale structural/architectural details for interior and/orexternal structural scenes. By combining 3D and 2D displays in asynchronized, coordinate-linked, true scale visual display, contextuallocation and spatial data are no longer mutually exclusive. Rather thecombined 3D/2D display uses scale information embedded in the header ofan image to enable a user to attain real time information that can bemanipulated simultaneously in 3D and 2D provides a unique state ofsituational awareness and intelligence of the environment being viewed.The simultaneous display of the 3D and 2D views are independent of 3Dand 2D window placement or size, fixed or floating.

The present invention also provides a measurement tool for use with anapplication suited for viewing a digitized drawing. The measurement toolcomputes lengths and areas (both regular shaped and irregular shapedareas) from a digital drawing in true scale. This is particularlyadvantageous with digitized architectural drawings or other drawingsthat are scanned from paper into a digital format where measuring orannotating the drawing in true scale is important and not easilymaintained over time or recaptured if it is lost.

In an embodiment, the present invention comprises the steps ofdigitizing a paper document, capturing the scale data and the physicalparameters of the paper being digitized (e.g., scanned), embedding thescale and physical parameter data in a header associated with the fileof the digitized image, and then storing the digitized image. Thepresent invention further provides for the processing of the header datawhen viewing the digitized image through a viewer application such thatthe header data can be used in calculating the true scale line lengthsand areas. For example, when the digitized paper being viewed is a floorplan, then the header data can be used to measure distances and areas onthe floor plan in true scale. Once the line is drawn, the true scalemeasurement is calculated using the header scale data, then it can befurther converted to a desired unit of measurement and then presented tothe user.

The step of capturing the scale and physical parameters of the paperbeing digitized comprises capturing the original scale information ofthe paper, the DPI of the scan, and the original size of the paper. Ifthe paper is imaged as a TIFF file, then the captured data is stored ina private tag of the TIFF header using TIFF header tags. It is known bythose of skill in the art that the TIFF header has both private andpublic TIFF header tags and that public tags are intended for aparticular or singular data type while private tags must be registeredto retain data for a particular purpose. For instance, public TIFFheader tags for file size cannot be used to store other data such asdescription data or scale data. Private tags are open fields and do nothave data that is intended to go in them unless they are registeredtags. Private tags can be used by software developers or be registeredto private companies so that particular tag can be used for onedesignated, well-defined purpose.

When viewing the TIFF image, a user can use the drawing tools that are apart of the viewer to draw a line or shape. The locations of the pixelsthat define the line or shape are captured by the viewer for use withthe header data to calculate the true scale measured length of the line.As mentioned above, the present invention provides for the measurementof lengths (for both lines and polylines) and areas (for both regularshapes as well as irregular shapes, such as rectangles, polygons andinverse polygons). Other tools employed with the TIFF image include aFind Shortest Path tool that in part uses the embedded scale informationin the TIFF header to calculate the shortest, fastest path between twouser chosen or dynamically updated points marked on the 2D map or in the3D rendering, the Find Shortest Path tool can also calculate andsimultaneously display multiple routes between two user chosen ordynamically updated points e.g. the Primary (shortest) route, theSecondary (next shortest) route, and Tertiary (optional) route to allowfor advanced contingency planning, a barrier tool that allows discreetpathways and entry/exit points to be manually or automatically marked asimpassable, stairwells to be marked for attack or evacuation, and havethose dynamic details trigger a recalculation of the shortest path. The3D Record Path tool records a path navigated on a 3D window andsimultaneously maps the path on the 2D display pane. Upon playback aDoor Detection Tool automatically tabulates environmentally orientateddoorways and passageways along an allotted path as a critical aid toemergency and response personnel operating in adverse conditions such asdark and smoked filled environments. Generated path can be stored andembedded in the digital file to facilitate planning, preparedness,simulated evacuations, and enhanced training.

In these changing times, it is imperative that in crisis situations,disaster response and the like emergency management personnel andbuilding personnel have access to a building's plan to better protectthe occupants, infrastructure and assets. What is needed is a system andmethod that gives emergency personnel the building architectural plansto scale in an interactive true-scale 3D and 2D visual environment, sothat they are useful to the emergency personnel and enable multiplein-context situational awareness data points to be experienced by systemusers at a remote location. This ability has historically been reservedonly for people within the physical structure itself. The system andmethod can be embodied in a software package.

The scale plans are useful to emergency personnel for planning ingressand egress routes for buildings or structures, including stadiums,arenas, bridges, tunnels, wharfs and the like. Additionally,point-to-point routing, manual or programmatic is easily determined.

The scale plans are useful to the public and emergency personnel forplanning ingress and egress routes both before and during an emergency.To prepare for possible emergencies, building tenants or management canuse the disclosed system to determine pre-arranged routes for enteringand exiting the building while viewing such detail simultaneously in 3Dand 2D. When an emergency occurs, emergency personnel can use theinvention's dynamic searching and delivery capabilities to determine, inreal time the routes to emergency exits. The system also allowsemergency personnel to determine multiple routes presented in ahierarchical shortest to longest format to and from a specific buildinglocation or area, and allows them to access or block specific portionsof the building in both 3D and 2D. The system allows the routes to bedetermined across multiple stories of a building, taking intoconsideration all human transportation infrastructures, e.g. buildingstairwells, and even from the exterior of a structure to any accessiblelocation on any floor of a structure.

The current invention facilitates point-to-point routing within astructure, allowing personnel to identify exact measured routes forreaching a specific location. Emergency personnel will know how to getfrom point A to point B, and the exact distance they must travel alongthe route. For example, when a building is engulfed in smoke, firepersonnel cannot see and must rely on other means to assess where theyneed to go. Utilizing this system, firefighters will know exactly howfar to go in any given direction to reach a location. Similarly, instadiums and arenas, security can utilize the disclosed system topinpoint problem areas and address security situations that may arise.Both emergency personnel and tenants or other people in the buildingwill be able to determine the location of emergency exits and routes tothe exits. Various routes to emergency exits can be determined in realtime using dynamic searching.

The invention can deliver the scale plan information in at least threeways. First, in one embodiment, the invention displays the informationon a computer monitor screen or display device and allows users to pickselected points or areas using a pointing device, such as a mouse,stylus or other user directed selection device. Second, the inventioncan display the information on hand-held devices that personnel cancarry. Third, the system can use a heads-up display, which displays therelevant information in a user's line of sight. Using the invention inthis manner would aid firefighters who often work in situationsinvolving reduced visibility. With the invention, a firefighter walkingin darkness can view a heads-up display that details any neededinformation, including current location and routes to a desireddestination.

At last, simultaneously 3D and 2D, to-scale displays can be used so thata planning board or planning personnel can determine access routes aswell as containment strategies or other strategies. Utilizing thedisclosed system, pre-arranged routes can be developed by buildingtenants or building management to determine ingress and egress routeswhich can increase fire safety preparedness, effectively for fire drillsand training. Further, for stadiums, arenas, and the like, security canutilize the disclosed system to pinpoint problem areas and determinesolutions to various security situations that may arise.

Users can implement the invention in at least three ways: (1) kiosks,(2) remote communication systems, and (3) an integrated system.

BRIEF DESCRIPTION OF THE DRAWING

Having thus described the invention in general terms, reference will nowbe made to the accompanying drawings, which are not necessarily drawn toscale, and wherein:

FIG. 1 is a schematic block diagram illustrating a system in accordancewith an embodiment with the present invention.

FIG. 2 is a flowchart of an embodiment of the present invention.

FIG. 3 is an illustrative user interface for inputting scale dataassociated with a scanned document, in accordance with an embodiment ofthe present invention.

FIG. 4 is an illustrative user interface for inputting data associatedwith a scanned document, in accordance with an embodiment of the presentinvention.

FIG. 5 is an illustrative user interface for viewing a scan document,wherein the user has drawn a line and the true scale measurement of theline is displayed to the user, in accordance with an embodiment of thepresent invention.

FIG. 6 is an illustrative user interface for viewing a scan document,wherein the user has drawn a polygon and the true scale measurement ofthe polygon is displayed to the user, in accordance with an embodimentof the present invention.

FIG. 7 is a schematic drawing illustrating the calculation of the lengthof a line, in accordance with an embodiment of the present invention.

FIG. 8 is a schematic drawing illustrating the calculation of the areaof a rectangle, in accordance with an embodiment of the presentinvention.

FIG. 9 is a schematic drawing illustrating the calculation of the areaof an ellipse, in accordance with an embodiment of the presentinvention.

FIG. 10 is a schematic drawing illustrating the calculation of thelength of a poly-line, in accordance with an embodiment of the presentinvention.

FIG. 11 is a schematic drawing illustrating the calculation of the areaof a polygon, in accordance with an embodiment of the present invention.

FIG. 12 is an illustration of a data display, in accordance with anembodiment of the present invention.

FIG. 13 is a flowchart of another embodiment of the present invention.

FIG. 14 is an illustrative user interface displaying a 3D/2Ddisplay/viewer.

FIG. 15 is an illustrative user interface displaying only the 3D windowof the display/viewer, as described in FIG. 13.

FIG. 16 is an illustrative user interface displaying only the 2D windowof the display/viewer, as described in FIG. 13.

FIG. 17 is an illustrative user interface displaying the applicationshell of the 3D/2D display/viewer, as described in FIG. 13.

FIG. 18A shows a Find Path Tool employed on a single floor shown on a3D/2D display/viewer.

FIG. 18B shows a path calculated using the Find Path Tool of FIG. 18A.

FIG. 18C shows a Find Path Tool employed on a multi-story building shownon a 3D/2D display/viewer.

FIG. 19 shows a Barrier Path Tool employed with a 3D/2D display/viewer.

FIG. 20 shows a Door Detection Tool employed with a 3D/2Ddisplay/viewer.

FIG. 21 shows a 3D Record Path Tool employed with a 3D/2Ddisplay/viewer.

DETAILED DESCRIPTION OF THE INVENTION

The present inventions now will be described more fully hereinafter withreference to the accompanying drawings, in which some, but not allembodiments of the invention are shown. Indeed, these inventions may beembodied in many different forms and should not be construed as limitedto the embodiments set forth herein; rather, these embodiments areprovided so that this disclosure will satisfy applicable legalrequirements. Like numbers refer to like elements throughout.

It will be appreciated that the systems and methods of the presentinvention are described below with reference to block diagrams andflowchart illustrations. It should be understood that blocks of theblock diagrams and flowchart illustrations, and combinations of blocksin the block diagrams and flowchart illustrations, respectively, may beimplemented by computer program instructions. These computer programinstructions may be loaded onto a general purpose computer, specialpurpose computer, or other programmable data processing apparatus toproduce a mechanism, such that the instructions which execute on thecomputer or other programmable data processing apparatus create meansfor implementing the functions specified in the flowchart block orblocks.

These computer program instructions may also be stored in acomputer-readable memory that can direct a computer or otherprogrammable data processing apparatus to function in a particularmanner, such that the instructions stored in the computer-readablememory produce an article of manufacture including instruction meansthat implement the functions specified herein. The computer programinstructions may also be loaded onto a computer or other programmabledata processing apparatus to cause a series of operational steps to beperformed on the computer or other programmable apparatus to produce acomputer implemented process such that the instructions that execute onthe computer or other programmable apparatus provide steps forimplementing the functions specified herein. Accordingly, blocks of theblock diagrams and flowchart illustrations support combinations of meansfor performing the specified functions, combinations of steps forperforming the specified functions and program instruction means forperforming the specified functions. It will also be understood that eachblock of the block diagrams and flowchart illustrations, andcombinations of blocks in the block diagrams and flowchartillustrations, can be implemented by special purpose hardware-basedcomputer systems that perform the specified functions or steps, orcombinations of special purpose hardware and computer instructions.

The present invention provides a measurement tool for use with a viewerapplication for viewing a digitized drawing. The measurement toolenables the measurement of horizontal and vertical lengths, distancesand areas (both regular shaped and irregular shaped areas) in truescale. While the present invention can be used with the digitalrepresentation of a paper document having a scaled drawing, such as anarchitectural drawings, engineering drawings or maps, it is describedbelow in the context of architectural drawings for illustrativepurposes. The disclosed embodiment should not be considered as limitingto the breadth of the invention.

The system also allows for an operator in a first location to provideinformation to a user at a second location. For example, once againusing the example of the firefighters above, the firefighters can have aheads-up display, for example on a visor of the firefighter's protectivegear, with the image of the building floor plan on his heads-up display.While supervisor on the street or at a central control location can thenprovide accurate structure navigation directions to the firemen via theheads up display, by oral instructions, or the like.

Another embodiment integrates all of a building's scaled plans into onesystem. Thus, structural, electrical, water, fire alarm, motiondetection, and other critical systems are all easily accessible toemergency personnel. The emergency personnel will have an integratedview of disparate data to effectively identify and locate hazardoussituations, potential victims, criminal perpetrators, or terroristelements.

The system could use standard RF communication, optic links, Bluetooth,IR links, or the like. Further, the three dimensional model can beintegrated with other building systems such as the intrusion alarm, firealarm, smoke alarm, electronic building management or electronicbuilding information management system so that various obstructions thatmay be present i.e., fire alarms, temperature caution monitors,hazardous materials locations, specialized building geometry andobstructions are taken into consideration when determining ingress andegress routes or other building centric decisions.

Further, emergency planning for other structures such as bridges andtunnels can be performed using the disclosed system. Additionally, GPSlocators can be used to track personnel location. In another embodiment,RF triangulation is used to determine exact personnel location. RFtriangulation is performed using antennas installed in a building or,for older buildings or buildings without such antennas, portabletriangulation units are used.

In another embodiment, the triangulation equipment is in emergencyresponse vehicles. RF triangulation can be used in conjunction with GPSlocators so that the triangulation points are known using GPS technologyand the exact location is determined by interpolation usingtriangulation.

The system uses existing electronic cad drawings or paper plans. Theplans are processed and entered into the system and stored in one ormore servers. The system, using a raster to vector conversion, preparesthe paper or legacy plans for use. The prepared plans are drawn toscale. Once entered into the system, the plans are immediatelyaccessible to all users, including remote users. In one embodiment, theplans are password protected.

The system can also be used for planning, decorating, and design. Oncethe plans are entered and scaled, other objects can be added to theplans such as furniture, rugs, and paintings. The system includes awalk-through and plan view feature so that the final layout can beviewed from a plurality of angles. Detailed measurements can be madeusing the disclosed system because the drawings are to scale.

In one embodiment of the system, kiosks are available in and around astructure that will allow users to select a start and end points. Thesystem then generates a three-dimensional depiction and two-dimensionalmap display of the route perfectly to scale. The produced image will beto scale so that the user will easily be able to determine distances.Further kiosks would allow end users to view details about a buildingsstructure and get contextual true scale intelligence about the userscurrent position in relation to the rest of the structure as the kioskslocation would be known and could be pre-mapped or plotted using thesystem. Or using the systems drawing and annotation tools the kiosk usercan display extended building data sets, for instance the location ofexits and or building fire equipment such as fire extinguishers, allperfectly to scale and in context to the underlying true scale digitalbuilding floor plan.

With reference to FIG. 1, an embodiment of the present inventioncomprises a scanner station 12, a database 14, a workstation 16, aprinter 18, a file input device 10, a transmitter 26, and a securitysystem central office 8. The scanner station 12 includes a scanner andassociated software required to capture a digital image of a paperdocument, such as a building blueprint, floor plan, riser diagram orother architectural or design drawing. In a preferred embodiment, thescanner station 12 comprises a high speed, large format scanner that isconnected to a desktop computer of sufficient speed and RAM to processlarge digital images. In one embodiment, the scanner utilizes eitherISIS or TWAIN interfaces, and that the compression/decompressionalgorithm utilized is TIFF CCITT Group 4, which is a losslesscompression algorithm. It is important that the algorithm be lossless topreserve the pixel-to-pixel bitmap data captured by the scanner. Thedatabase server 14 comprises any suitable database for storing the imagefile created by the scanner and its associated software. In anotherembodiment, the image files are input into the database as digitalfiles, e.g., cad files and the like. The database stores entire floorplans and structural details for a complete facility, making the dataimmediately accessible. Thus, emergency crews are fully aware of theentire building layout and any potential trouble spots well in advanceof entering the structure.

The workstation 16 may be any suitable computing device with userinterface means such as a monitor, keyboard, mouse, stylus, etc. Theworkstation may be a desktop computer or a portable computing device,such as laptop 28 a, PDA 28 b or cell phone 28 c capable of displayingtwo and three dimensional images. The workstation includes a viewer 240.In the illustrated embodiment, the viewer 240 is a TIFF viewer capableof reading (i.e., decompressing) a TIFF image and displaying it to auser. The viewer 240 can be built, for example, utilizing the viewercomponents and tools provided by LEAD Technologies, Inc. Specifically,LEAD Technologies, Inc. provides a decompression tool, rubber band tool,display tool, overlay display tool, overlay storage tool and tag readtool that can be assembled into a TIFF viewer. A key aspect of theviewer 240 is the inclusion of a measurement calculator 22, inaccordance with the present invention, for calculating the true scalemeasurement of lines and shapes drawing with the viewer 240.

The printer 18 is any suitable printer capable of printing from theworkstation 16, and a network 24 interconnects the aforementioneddevices. The network 24 may comprise any telecommunication and/or datanetwork, whether public or private, such as a local area network, a widearea network, an intranet, an internet and/or any combination thereofand may be wired and/or wireless. Due to network connectivity, variousmethodologies as described herein may be practiced in the context ofdistributed computing environments.

In one embodiment of the invention, the workstation 16 has one or moredocking stations associated with it. These docking stations are used todownload the floor plans and structural details to a device such as atablet PC 28 a, PDA 28 b, cell phone 28 c, and the like. Thus, inaddition to being able to having a printout of the data, an electroniccopy can be used. In another embodiment, the data is transmitted to aPDA, cell phone, or the like utilizing transmitter 26. In oneembodiment, data is transmitted to a heads-up display using Bluetoothtechnology, or the like.

In practice, the transmission of the data files to the cell phone, PDAor the like is done utilizing existing cell phone, wi-fi and pagerinfrastructure. In yet another embodiment, the data can be transmittedon standard FM signals s or any wired or wireless network capable oftransmitting data packets.

With reference to FIG. 2, a method in accordance with the presentinvention is shown. As an initial step, a paper document is digitized,as indicated by block 58. This step includes scanning the paper documentusing the scanner station 12 to create a bitmapped raster image or usingan input device to load a digital file. In the illustrated embodiment,the paper document is a drawing. The scale data and physical parametersof the paper drawing being scanned are captured and associated with thebitmapped image. Specifically, the original scale information of thepaper drawing, the DPI of the scan, and the original size of the paperare recorded and embedded within the digital image. In anotherembodiment, the input is from file input device 10, which inputs adigital file.

An illustrative user interface for recording this information isprovided in FIG. 3, which shows a Master File Info window 32 forentering the scale and physical parameter data of the paper drawingbeing scanned. Of particular interest, the X-DPI and Y-DPI fields 34,36are where the direct optical scan characteristics of the scanner thatare utilized for the scan are recorded. These values should becalibrated to ensure their accuracy. The SCN Width and SCN Height fields38,40 are the actual pixel dimensions of the scanned image. The Scalefield 42 is where the actual scale of the drawing is recorded as aninteger. The value inputted may be calculated using the Scale Finder 44,which is provided at the selection of the Scale Finder button 46. Theuser merely enters the scale from the drawing in the correct units, andthe Scale Finder will write the correct scale value into the Scale field42. For example, if the scale was one inch equals three feet, the ScaleFinder would write 36 into the Scale Field 42. Similarly, if the scalewere one centimeter equals one meter, the Scale Finder would write 100in the Scale field 42.

It should be noted that the information recorded and associated with thedigital image file does not necessarily have to be recorded at the timethe image is scanned or otherwise acquired. Also, additional informationidentifying the paper document may also be recorded, such as thebuilding name, building owner, date of drawing, exact geospatiallocation, i.e. latitude and longitude, etc., as shown in the userinterface 50 of FIG. 4.

In the illustrated embodiment, the paper drawing is optically scannedand saved as a TIFF file, and the captured data is stored in the TIFFheader using TIFF header tags. TIFF Tag 50271 is a suitable location forstoring the scale and physical parameter data. A suitable data structurefor such information may be:

Tag 50271=DBSWWWWHHHHAABBSSSSSSSDB

DBS=Digital Building Plan Tag (letters “DBP”)

W=Width (Original image scan width in pixels)

H=Height (Original image scan height in pixels)

A=HDPI (Horizontal DPI of scan)

B=VDPI (Vertical DPI of scan)

S=Scale (Inches to Inches document Scale, i.e., 1″=36″)

DB=Digital Building Identifier Tag (“DB”)

The Adobe tag 50271 is stored as ASCII data type with a variable lengthof 24 characters beginning with either “DBS” and ending with the DigitalBuilding Identification Tag “DB”. The width W is the scan width of theimage in pixels. The height H is the height of the image in pixels. TheA and B are the horizontal and vertical direct optical DPI of thescanner, respectively. This is the direct optical resolution of thescanner. The scale S is the scale taken from the paper drawing.Alpha-numeric ASCII characters with ASCII values between #48 and #90 maybe used in data fields to avoid data and compression conflicts. In theillustrated embodiment, the values are converted to a base 34 number

Referring back to FIG. 2, once the digital image file has been created,it may be stored, as indicated by block 60, preferably within a RAIDserver or SAN with its accompanying entry in the database sever 14.However, the digital image file may be stored in the memory of virtuallyany computing device, including at the scanning station 12, workstation16, or a cell phone 28 c, PDA 28 b, or the like. In a preferredembodiment, the plurality of digital image files are stored together ata central data repository

The digital image may then be viewed by a user, as indicated by block62, preferably at a workstation 16. The digital image file is sent tothe workstation via the network 24. The digital image display/viewer240, can be utilized to open and view the digital image. The digitalviewer application should at a minimum, have some drawing tools, with atleast the ability to draw lines and to interconnect those lines to forma shape.

The user then utilizes the display/viewer to draw a line or shape (e.g.,a regular shape or irregular shape, such as a polygon or an inversepolygon) or to map ingress and egress routes or calculate distances asindicated by block 64. The true scale measurement of a line or thelength of distance of a route is calculated and presented to the user asindicated by block 66. For example, as illustrated in FIG. 5, the userhas drawn a line 70, such as by the clicking and dragging the mouse ordragging a stylus. The true scale measurement of that line is calculatedand presented to the user in the tool bar field 72, as indicated byblock 66. Another example is provided in FIG. 6, wherein the user hasdrawn a polygon 74 and the true scale area of the polygon is presentedto the user in the tool bar filed 76. Thus, in accordance with thepresent invention, the digital image viewer 240 is modified to accessthe scale and physical parameter information embedded within the digitalimage and calculates the true scale measurement of a line or area of ashape.

In the illustrated embodiment, the digital image viewer 240 reads theTIFF header tag 50271 to retrieve the scale and physical parameter data.The digital image viewer 240 then provided the measurement calculator 22with the pixel data defining the user's drawing (e.g., a line or shape)and the scale information read from the digital image header tag. Themeasurement calculator 22 then calculates the true scale measurementusing that information and the pixel location data of the line or shape.The calculated measurement can be presented to the user in any suitableformat or location on the screen, though in the illustrated embodiment,the measurement is presented in a tool bar at the bottom of the window.

For illustrative purposes, several calculations are provided for lengthsand areas of annotations drawn by the user using the drawing tools ofthe digital image viewer 240, and in particular, using a mouse inputdevice.

The length of a line 80 is calculated with general reference to FIG. 7.The user initially triggers the calculations with a mouse-down event(while the line annotation is selected from the drawing tool bar). Thisevent provides the first point of reference (X₁, Y₁) in pixels, asillustrated in FIG. 7. When the user releases the mouse button thistriggers a mouse-up event. This event provides the second (and final)point of reference (X₂, Y₂) in pixels. With these two points ((X₁, Y₁)and (X₂, Y₂)) measurement calculator 22 can calculate the length betweenthem (in pixels) using the Pythagorean Theorem, as provided by Equation(1) below:Length (in pixels)=((x ₂ −x ₁)²+(y ₂ −y ₁)²)^((1/2))  (1)

This length is then divided by the resolution of the image to producethe representative length in inches on the original plan, or drawing, asprovided by Equation (2) below:Length (in inches)=(length (in pixels))/(image resolution (dpi))  (2)

This length (in inches) is then multiplied by the blueprint scale(embedded into the header of the TIFF image) to produce the actuallength (in inches) of the line, as provided by Equation (3) below:Actual length=plan length (in inches)×plan scale  (3)

The measurement calculator 22 then provides this true scale measurementto the viewer 240 for display to the user. If desired, furthermeasurement conversions can be performed to calculate any unit ofmeasurement desired. For example, measurement units can be convertedfrom inches to feet or meters by simple multiplication of the unitconversion factor.

Next, the area of a rectangle 82 will be calculated with reference toFIG. 8. Initially, the user triggers the calculations with a mouse-downevent (while the rectangle annotation is selected from the drawing toolbar). This event provides the first point of reference (X₁, Y₁) inpixels. When the user releases the mouse button this triggers a mouse-upevent. This event provides the second (and final) point of reference(X₂, Y₂) in pixels. With these two points ((X₁, Y₁) and (X₂, Y₂)), themeasurement calculator 22 can calculate the area between them (inpixels) using the Pythagorean Theorem, Equation (4) provided below:Area (in pixels)=(x ₂ −x ₁)²+(y ₂ −y ₁)²  (4)

This area is then divided by the squared of the resolution of the imageto produce the representative area in inches on the original plan, ordrawing, as provided by Equation (5) below:Area (in inches)=(Area (in pixels))/(image resolution (dpi))²  (5)

This area (in inches) is then squared and multiplied by the square-rootof the blueprint scale (embedded into the header of the TIFF image) toproduce the actual area (in inches) of the selected rectangle, asprovided by Equation (6) below:Actual area=(plan area (in inches))² (plan scale)^((1/2))  (6)

The measurement calculator 22 then provides this true scale measurementto the viewer 240 for display to the user. If desired, furthermeasurement conversions can be performed to calculate any unit ofmeasurement desired. For example, measurement units can be convertedfrom inches to feet or meters by simple multiplication of the unitconversion factor.

The area of an ellipse 84 is illustrated next with general reference toFIG. 9. The user initially triggers the calculation with a mouse-downevent (while the ellipse annotation is selected from the drawing toolbar). This event provides the first point of reference (X₁, Y₁) inpixels. Then the user releases the mouse button this triggers a mouse-upevent. This event provides the second (and final) point of reference(X₂, Y₂) in pixels. With these two points ((X₁, Y₁) and (X₂, Y₂)), themeasurement calculator 22 can calculate the area between them (inpixels) using the Pythagorean Theorem, Equation (7) provided below:Area (in pixels)=Π[((x ₂ −x ₁)/2)+((y ₂ −y ₁)/2)]  (7)

This area is then divided by the squared of the resolution of the imageto produce the representative area in inches on the original plan, ordrawing, as provided by Equation (8) below:Area (in inches)=(Area (in pixels))/(image resolution (dpi))²  (8)

This area (in inches) is then squared and multiplied by the square-rootof the blueprint scale (embedded into the header of the TIFF image) toproduce the actual area (in inches) of the selected ellipse, as providedby Equation (9) below:Actual area=(plan area (in inches))² (plan scale)^((1/2))  (9)

The measurement calculator 22 then provides this true scale measurementto the viewer 240 for display to the user. If desired, furthermeasurement conversions can be performed to calculate any unit ofmeasurement desired. For example, measurement units can be convertedfrom inches to feet or meters by simple multiplication of the unitconversion factor.

The length of a poly-line 86 is calculated next with general referenceto FIG. 10. The user initially triggers this calculation with amouse-down event (while the poly-line annotation is selected from thedrawing tool bar). This event provides the first point of reference (X₁,Y₁) in pixels. The user then moves the mouse and clicks (theleft-button) to add additional nodes [(X₂, Y₂). (X₃, Y₃), . . .(X_(n+1), Y_(n+1))]. Once the user is completed with the poly-line theycan either double-click the left mouse button or single click the rightmouse button to end the poly-line and trigger the calculation of thelength. This provides, for use in the calculation of the length, (n+1)nodes and (n) line segments; where ‘n’ is some arbitrary absolutenumber. With this collection of points the measurement calculator 22 cancycle through each node and calculate the summation of the lengths ofeach line segment using the Pythagorean Theorem (on each segmentrespectively), as provided below by Equation (10):

$\begin{matrix}{{{Length}\mspace{14mu}\left( {{in}\mspace{14mu}{pixels}} \right)} = {\sum\limits_{i = 1}^{n}\left( {\left( {x_{i + 1} - x_{i}} \right)^{2} + \left( {y_{i + 1} - y_{i}} \right)^{2}} \right)^{({1/2})}}} & (10)\end{matrix}$

This length is then divided by the resolution of the image to producethe representative length in inches on the original plan, or drawing, asprovided by Equation (11) below:Length (in inches)=(length (in pixels))/(image resolution (dpi))  (11)

This length (in inches) is then multiplied by the blueprint scale(embedded into the header of the TIFF image) to produce the actuallength (in inches) of the poly-line, as provided by Equation (12) below:Actual length=plan length (in inches)×plan scale  (12)

The measurement calculator 22 then provides this true scale measurementto the viewer 240 for display to the user. If desired, furthermeasurement conversions can be performed to calculate any unit ofmeasurement desired. For example, measurement units can be convertedfrom inches to feet or meters by simple multiplication of the unitconversion factor.

The area of a polygon 88 is next illustrated with reference to FIG. 11.The user initially triggers these calculations with a mouse-down event(while the polygon annotation is selected from the drawing tool bar).This event gives us the first point of reference (X₁, Y₁) in pixels. Theend user then moves the mouse and clicks (e.g., the left-button) to addadditional nodes [(X₂, Y₂). (X₃, Y₃), . . . (X_(n+1), Y_(n+1))]. Oncethe user is completed with the polygon they can either double-click theleft mouse button or single click the right mouse button to end thepolygon and trigger the calculation of the length. This provides, foruse in the calculation of the length, with (n+1) nodes and (n) linesegments; ‘n’ is arbitrary and absolute. With this collection of pointsone can iterate through the line segments and get a running total forthe area. This area is calculated by first identifying a baseline belowthe polygon, then identifying a trapezoid whose sides consist of (1) asingle line segment on the polygon, (2) a line from the rightmost pointin the polygon segment to the baseline which is perpendicular to thebaseline, (3) a segment of the baseline, and (4) a line from thebaseline to the leftmost point in the line segment (drawn perpendicularto the baseline). The area of the trapezoid is calculated with Equation(13) below:

$\begin{matrix}{{{Area}\mspace{14mu}\left( {{in}\mspace{14mu}{pixels}} \right)} = {\left( {1/2} \right){\sum\limits_{i = 1}^{n}\left( {{x_{i}y_{i + 1}} - {x_{i + 1}y_{i}}} \right)}}} & (13)\end{matrix}$

This area is then divided by the squared of the resolution of the imageto produce the representative area in inches on the original plan, ordrawing, as provided by Equation (14) below:Area (in inches)=(Area (in pixels))/(image resolution (dpi))²  (14)

This area (in inches) is then squared and multiplied by the square-rootof the blueprint scale (embedded into the header of the TIFF image) toproduce the actual area (in inches) of the selected rectangle, asprovided by Equation (15) below:Actual area=(plan area (in inches))² (plan scale)^((1/2))  (15)

The measurement calculator 22 then provides this true scale measurementto the viewer 240 for display to the user. If desired, furthermeasurement conversions can be performed to calculate any unit ofmeasurement desired. For example, measurement units can be convertedfrom inches to feet or meters by simple multiplication of the unitconversion factor.

The present invention permits the user to view the file in an emergencysituation. For example, if firefighters are dispatched to a burningstructure, the firefighters download the digital files to a PDA or thelike so that they have the entire structural layout of the building. Inone embodiment, a first user at a workstation provides routing or otherinformation to a second user in a structure. The second user receivesthis information on a PDA, cell phone, tablet computer, heads-updisplay, or the like.

In one embodiment, the user views the drawing on a viewer 240 such as acomputer, laptop 28 a, PDA 28 b, or the like. The blueprint presented onthe PDA provides the user (emergency response personnel) with accuratemeasurements of floor space and distances between entrances, exits andtarget locations. Additionally, the system provides full scalingfunctionality. This scaling functionality allows a user to zoom in andout of a specific area to provide as much or as little detail asrequired. In one embodiment, to zoom a user uses a zoom tool to selectthe area that should be zoomed. Alternately, the system will zoom inpreset increments, i.e., 10%, 20%, 30% around a specific area merely bytapping a stylus in the desired zoom area. It should be noted that nomatter how much a user magnifies the display, it remains accuratelyscaled.

Along with measurements, the system can display, visualize and calculatedetails about other structural elements such as stairwells, elevators,entrances, exits, shaft ways, building management systems, coolingunits, emergency power; emergency command posts, areas of refuge and thelike. Further, the location of sprinklers, fire extinguishers, hosehook-ups, risers, HVAC systems and electrical access panels can also beprovided on the layout. In yet another embodiment, hazardous materialscan also be displayed.

In one embodiment of the invention, a building security system is tiedinto the network. The security system can provide such data as activealarms such as fire alarms, smoke alarms, carbon monoxide alarms, smokealarms, and the like. In this manner, emergency workers can determineproblem areas and potential rescue situations. Additionally, abuilding's motion sensors can be tied into the network such that peoplein the building can be tracked, thereby enabling enhanced rescueattempts. For instance, GPS locators can be used to track people andequipment. Alternatively, if a hostage situation exists, police can usethis data to plan a terrorist mitigation or asset recovery mission.

FIG. 12 is an illustration of a data display, in accordance with anembodiment of the present invention. As shown, the display is zoomed into so the user can discern a desired level of detail. In a preferredembodiment, a cursor is used to select a start point such as entryway116 and end point 110. The system programmatically calculates a routefrom 116 to 110. Two routes are shown in FIG. 12. A first route, 102, isshown from the entryway 116 to a point 110 in a back office. A secondroute 104 is shown from the entry point 116 to a utility closet housinga PBX and Hub. In one embodiment, items such as outlets 114, switches118, and telephone jacks are shown. Other items such as electricalconduits, HVAC systems, and plumbing are shown. The display providesdata from motion sensors 100, heat and smoke alarms, and door and windowsensors, which are tied to the display.

In one embodiment, a kiosk 120 is present. Building visitors use thekiosk 120 as a guide. In one embodiment, patrons use the kiosk as adirectory. Patrons either selects a destination graphically, e.g., adesired office 110, or selects from a directory listing. Either way, theroute, and if desired route adjacent areas of interest such as abuilding locations or hazard, are displayed.

Users use zoom tool 108 a user zooms in and out of a specific area toprovide as much or as little detail as required. In one embodiment, tozoom, zoom tool 108 selects the specific area to be magnified.Alternately, the system will magnify in preset increments, i.e., 5%,10%, 15%, etc. using the selected area as the center of the area to bemagnified. In another embodiment, the preset increments are selectableby the user. It should be noted that accurate scaling of the image andaccurate scaling of all measurements are maintained at eachmagnification point.

As described above, the system and method according to the presentinvention provides the ability to take paper based original drawings andprovide scaled digitized images that allow for accurate point to pointmeasurement and routing. The foregoing embodiments are given by way ofexample for the purpose of teaching the method and system of the presentinvention. The present invention is not limited to these embodiments andone skilled in the art may affect various changes and modificationwithin the spirit of the invention as defined in the appended claims.

Also as mentioned above, the present invention can be used by anyoneincluding and not limited to firemen, emergency response, command andcontrol, police, EMT, utility workers, military, and buildingoperations, management tenants and ownership as well as facilityengineers. In one embodiment the invention can be implemented for a cityemergency operations center with access being granted to local, federaland state fire, police and emergency services users.

Another method in accordance with the present invention is shown now inFIG. 13. At block 200 a document is scanned or otherwise digitized andthe original document image scale information, DPI of the scan andoriginal paper size is captured and embedded into the digital fileheader of a two dimensional digital raster image. The scanned documentmay include floor plans from a single building, or more likely acollection of buildings.

A true scaled three dimensional virtual digital model rendering (alsocalled the 3D digital image or 3D digital rendering) is created based onthe two dimensional digital raster image. Specifically, the viewer andthe associated tool sets are used manually, or through programmaticsteps, to annotate the 2D digital raster image so that the propertiesand positions of the 2D annotations are programmatically translated tocreate a scene graph which is then used to create the associated-3Ddigital image. See block 205. The scene graph lists the objects,properties, and transforms that describe the 3D digital image. The scenegraph is organized by loose groups of similar object types rather thanany specific order of objects.

Once the 2D digital raster image file and the 3D virtual model fileshave been created, each file may be organized and stored as individualyet associated files using a file system in the database sever 14, thememory of any computing device or a central data repository. See block210. Individual documents are stored in a file server and associated todatabase records. Documents may preferably be organized in the databaseby building and floor or some universal standard.

The user may search the database for the digital image file to be viewedat a workstation 16 using a computer or laptop 28 a or the like. Seeblock 220. Users of the database can locate and view individual digitalplans or groups of digital plans. The database can be located on aclosed network, a web accessible network or a localized computing devicewith no network connectivity.

The user can query the database to locate a specific digital image orgroup of digital images (block 230) such as an entire 3D building, adigitized floor plan document or the individual floor 3D scene. Theselected digital image file is then distributed to the workstation viathe network 24 and viewed, to-scale on a dual 3D/2D digital imagedisplay/viewer 240. The digital viewer 240 may include but is notlimited to synchronized graphic rendering devices, synchronized userinteractive graphic displays, linked graphic representations, real timeevent linked display mechanisms, synchronized horizontal displaysurfaces, synchronized holographic displays, synchronized graphicscreens, dual monitor heads up displays, auto stereoscopic displays andimmersive graphic environment. In one embodiment, the digital image fileis pre-populated onto mobile computer systems with 3D enabled videographic hardware and software.

The digital image display/viewer 240 can open and render the digitalimage files and retrieve the original image/document scale informationthat is embedded in the 2D digital image header. The digital imagedisplay/viewer 240 has a multiple document interface having displaymeans, windows or view ports that are linked and coordinated and can beseen simultaneously.

An illustrative user interface of the display/viewer 240 is shown inFIG. 14 comprising an application shell 243, two document view ports (a3D window 242 being 3D graphics enabled and a 2D window 244 being 2Dgraphics enabled), a basic set of drawing tools and a menu interface toactivate functionality and interact with the 3D and 2D displays. Boththe 3D window 242 and 2D window 244 can be sized and positioned to theuser's preference. In one instance, the 3D window 242 occupies 40% ofthe left hand side (LVP) of the display/viewer 240 and the 2D window 244occupies 40% of the right hand side (RVP) of the screen.

Images displayed on the 3D display 242 are true scaled three dimensionaldigital renderings that are initially viewed as a ground plain viewwherein the user perspective is parallel to the plain of the documentsurface. The view angle can be manually changed and rotated on the X, Yor Z axis permitting the user a view perspective at any desired angle orheight. See FIG. 14. Images displayed on the 2D window 244 are viewedperpendicular to the surface of the document. For instance a scaleddigital floor plan or an architectural drawing view in 2D will be viewedin plan view which is maintained when the document is zoomed or rotatedor otherwise manipulated. The 2D digital raster image is the masterfile, real data, and is the origination point for data as scale data isembedded and read from the digital image rendered in this location forboth the 3D and 2D windows.

The simultaneous viewer 240 serves as a tool that provides the user withan enhanced awareness of a situation or environment and an overallintelligence of the structure shown in the drawings. The user canmanipulate the digital images using the simultaneous viewer 240 usingtools employed with the viewer 240. For instance, the user can identifystart and end points on an image displayed in a select window (block250), use the shortest path tool to find the fastest route between twographically marked/user chosen points with no restrictions orlimitations on the end and start point locations in context to thebuilding image (block 260), select an stairwell and set it to be anEvacuation stairwell which will then only allow that stairwell to beused for routes calculated from a building floor down through thestructure and/or an attack stairwell which will then only allow thatstairwell to be used for routes calculated from a building location upthrough the structure and force the Shortest Path tool to use thatstairwell as a first node in the route calculation (block 265),calculate the shortest path between points and display the totaldistance using the shortest path algorithm and the embedded scaleinformation (block 270), graphically display the shortest pathdiagrammed in the 3D window 242 (block 280) and 2D window 244 (block285) and also view the true scale measurement of the shortest calculatedpath (block 290). Additionally the calculated path data, i.e. thegraphic display data and the true-scale path measurement data can beselected in the 3D window and with a mouse command and an applicationdialog appears that allows the calculated path to be named and saved forfuture retrieval and use (block 295).

Creating the Two Dimensional Digital Raster Floor Plan Image

Referring back to block 220, the scanned document creates a true-scale2D digital image, which is displayed in the 2D window 244. The capturedscale data is embedded in the TIFF header of the 2D digital image. Eachobject represented in the 2D digital image of the 2D window 244 isrepresented by a 2D primitive, usually a line for a wall, or a rectanglefor a door, window, or other object. The shapes represented by thedigital image in the 2D window 244 may be either 2D only data (beingdata that does not have 3D-associated representations) or data with3D-associated representations, also called 3D related objects. The3D-related objects have additional information relating to type,orientation, off-ground position and object height that is notrealistically presentable in the 2D window.

Creating the True Scaled Three Dimensional Virtual Digital Model Image

As mentioned above, the scale information embedded in the originalraster 2D floor plan is also used to construct a true scaled threedimensional digital model image, specifically the scale information istranslated into associated 3D virtual models via the scene graph. The 3Ddigital image/3D digital rendering displayed in the 3D window 242 isprepared using the 3D-related objects data graphically overlaid on theoriginal 2D digital image displayed in 2D window 244. Each object in the2D data set is evaluated to extract information contained in 3D relatedobjects and any predefined model associated with the object is added tpthe scene graph. (2D Data annotation objects representing walls do nothave separate associated models. The 2D wall annotations are directlyextruded into true scale 3D geometry elements.) Then the 3D objects arearranged using positional coordinate information, 3D object datastructures are created, and pointers to the 3D structures are added tothe associated 2D object data sets.

Using the 3D digital image, a user can use a computer mouse to manuallynavigate the 3D virtual model of the building and virtually walk throughrooms and hallways of the model displayed on the 3D window 242. A 3Dwindow camera is used with the 3D window 242 to assist in viewing thepath taken in the 3D virtual model. The 3D window camera can be set in alocation to define the view of the structure that is presented in the 3Dwindow 242 and render the scene from the camera's point of view. Thepath taken in the 3D virtual model can be viewed as if the virtualmoving camera is positioned on a person's head.

The display/viewer 240, the 3D window 242 and 2D window 244 are designedto work in cooperative form. For instance, the 3D window 242 and the 2Dwindow 244 may be child applications of the application shell (a parentapplication) 243, or two separate applications may be tied to a thirdapplication, or all three applications are tied together. As a result,using the display/viewer 240 the user can not only simultaneously viewthe 2D digital image of a floor plan in the 2D window 244 and theassociated 3D digital model image of the structures and objects of thefloor plan 242 but also synchronize, in real-time an action taken in onewindow and display it in the other window. Furthermore, using the toolsemployed with the display/viewer 240 the user can manipulate or deriveinformation from the digital images. As an option, the present inventionallows the user to open one window, 242 (see FIG. 15) or 244 (see FIG.16), and manipulate the digital image using only the open window alone.All changes or other input made in the open window is tracked andrelayed to the closed window when it is opened again, for instance, bythe parent application 243 (see FIG. 17) when the parent application isused to manage all changes made in the 3D window 242 and 2D window 244.

The 3D window 242 and 2D window 244 of the display/viewer 240 are trueto scale, coordinate matched, and linked in real time providing aninteraction between the images in each the 3D and 2D windows 242, 244.The mechanism by which the objects are kept accurately synchronized isaccomplished by associating scale, selection events and coordinates fromone window 242, 244 to the other window 244, 242, respectively.

Synchronization Using Scale

As described above, the scale information of the original image/documentis embedded in the digital image header. This data is retrieved and usedby the digital image viewer 240. As the 3D digital image is associatedto and based on the 2D digital image, the true scale value is the samein the two windows 242, 244. In some embodiments the scale between thewindows 242, 244 could be different. In such cases an algorithm may beused to translate the scale from one window to the other.

Synchronization Using Selection Events

Selection event synchronization takes advantage of the fact that eachobject movement or position change (user input) creates a softwareselection/transform event. These selection events provide the identityand updated position for the object. Depending on which window the useremploys, the selection events or action taken by the user in one windowis simultaneously repeated (causes a re-draw) in the other window.Generally, when an object is moved, the selected object is flagged, a 2Dselection event list is cataloging each flagged/selected object (loggedselection event) and an update is affected using the 2D selection eventlist in the corresponding window. Variations are present in this processdepending on which window is used to create the selection event.

Changes Made to Objects in the 2D Window

When a user makes a change to one or several objects in the 2D window244, a flag is set by the selected object and the flagged object is putin a 2D selection event list. An Idle State Processing Program, is partof the parent application, is then employed to update the 3D digitalimage. For instance when deleting an object from the 2D window the IdleState Processing Program will cycle through the 2D selection event listchecking the flagged objects and deleting the corresponding 3Drepresentation of the flagged 2D object in the scene graph. The 3D datais deleted during an update of the 3D digital model image as it istranslated from the real 2D data.

The 2D selection event list containing the updated information is theniterated through and new 3D objects are generated and placed into thescene graph. Accordingly, to effect a change to an object's 3Drepresentation, it must be located in the scene graph, its data updated,and the scene graph re-rendered. The replacement of the objectsstraightforward and efficient given that the scene graph is organized asloose groups of similar object types. The scene graph can be manipulatedso that updates do not cause the entire scene to re-render only theaffected objects for which a selection event is recorded.

To prevent thrashing (causing too many re-renders per object move) theIdle State Processing Program is only initiated when an idle state inthe application is detected by the system, such as when there is noselection event, change or input from the user. Here, when the IdleState Processing Program is opened, changes are made to the scene graphto update the 3D digital image. Unnecessary changes are furthercontrolled by keeping track of which objects have been selected. Boththe 3D and 2D data sets contain lists of the currently selected objects,and it is only the objects in these smaller lists that will be changed.Accordingly, once the 3D digital image has been updated, flags are setto prevent the 3D-side-change selection events from triggering changeson the 2D side.

Changes Made to Objects in the 3D Window

When a user moves or rotates an object in the 3D window 242 a selectionevent is triggered, a flag is placed by the 3D selected object in thescene graph and a 3D selection event list is created. The selectionevent occurring in the 3D window 242 possesses the identity of theobject and transform information for the action taken. The transform isthe new X, Y and Z positional and rotational angle information that hasbeen applied to the 3D object. The transforms are applied to only theselected 2D object(s) in the 3D selection event list. Unlike the changesinitiated in the 2D window 244, here, the selected 2D object data is notdeleted but rather is directly modified because 2D data is treated asthe real or source data.

The selected 2D data is updated using known scaling factors and positionand orientation information needed to correctly show on the 2D digitalimage or representation in the 2D window 244. Additional informationsuch as z-axis rotation or vertical position information cannot be shownin the 2D window, but is saved in the object's data set.

Here too, changes are made in the background of the 2D digital image toeffect an update after the Idle State Processing Program detects noactivity and flags are set that prevent the already updated 2D digitalimage components new positions from triggering further 3D selectionevents.

Synchronization Using Coordinates

The coordinate matched feature of the dualthree-dimensional/two-dimensional visual display/viewer 240 is based ona non-rendered mathematically accurate grid system. Each of the 3D and2D digital images has a non-rendered grid positioned on top thereonreflecting the precise pixel dimensions of the document. The coordinatesystem of each window in the display/viewer 240 is linked to a commonreferenced point or cross referenced point usually being the upper-leftcorner of the floor plan as the 0, 0 point.

Thus, a 2000 W×1500 H pixel document has a corresponding 2000×1500 spacegrid on top of it. This grid is a computed virtual entity that is notdrawn or seen and can extend infinitely on any axis. When viewing theentirety of the document at 0% zoom, the upper left hand corner of thedocument is the 0 Y (Vertical Axis), 0 X (Horizontal Axis) and, whenapplicable, 0 Z (Three-Dimensional axis).

The linking of coordinate positions for data objects displayed in eachthe 2D window 244 and 3D window 242 of the display/viewer 240 isdependent on the non-rendered grid. A two-way relationship is created bythe linking of the coordinate systems. Thus when an object is moved orannotations are made, manually or programmatically, in the 3D window242, the corresponding 2D graphic representation in the 2D window 244will move near simultaneously and vice versa.

The coordinate system inherent in the display/viewer 240 enables linkingby allowing a discreet set of pixels to be colored graphically to mark achosen shape, line or point. Indeed a point marker could be as small as1 pixel W 1 pixel H. Because pixels have a definitive size that cannotbe divided graphic representations of lines and shapes are alwaysestimations.

The coordinate system allows relative positions to be defined andcorresponded to other relative or arbitrary positions. The coordinatesystem of the present invention may even correspond to real worldlocations defined by coordinates of outside systems such as geographiccoordinate system including latitude and longitude, e.g. a GIS systems.The display/viewer 240 can receive coordinate input from a third oroutside coordinate system as long as the external coordinate grid andscales can be converted to the coordinate system shared by the windows242, 244. Such conversion occurs by translating the disparate coordinatesystem scales and defining at least one common reference point.Preferably multiple common reference points would be established.

By using outside coordinate systems, the present invention allows usersto receive coordinate data describing the location of an asset or objectbased on the Geographic Coordinate system and translate thosecoordinates to an exact spot on the 3D and 2D windows 242, 244representations of a structure.

In practice, a point defined graphically in the 2D window 244 can becalculated to have a certain coordinate position because of the grid anda relative true scale measured size resulting from scale data that isembedded in the digital image header.

The present invention combines contextual and structurally connectedvisual information to allow the user to obtain an enhanced situationalawareness of a structure and to intelligently navigate through thestructure. For instance, users can simultaneously see their immediatetrue scale spatial environment in the 3D window 242, (along with whatthe room looks like) and their accurate, true scale orientation (withrespect to the end point/goal) and current position in the building asshown in the 2D image displayed on the 2D window 244.

The coordinate-linked, dual and simultaneous nature of the 3D/2Ddisplay/viewer 240 when combined with the tools of the viewer 240 allowthe user to interact with the digital images in each window 242, 244 andthus better understand an environment. The user can manually trace out aroute, experience (virtually walkthrough) the route, describe theroute's environment during the virtual walkthrough, see a real timeposition indicator for the assets location in the 2D view port, andobtain the true scale measurement between objects and locations or ofthe navigated path displayed and saved. Also, once the 3D visualnavigation of the structure can be recorded, it can be madedistributable using standard video files outputs.

Tools

The tools employed with the display/viewer 240 include but are notlimited to a basic scale measurement calculator 22, a basic graphicdrawing tool set capable of making precise colored annotations on thedigital image, a Find Shortest Path Tool 300, a Path Barrier Tool 360, aDoor Detection Tool 330 and an Evacuation Simulation Tool. These toolsmay be managed by the parent application 243 and can be used in eitherwindow 242, 244.

Drawing Tools

As described above, the drawing tools 410 permit a user to constructlines, shapes and points with vector or raster graphic drawings. Thesedrawings are then presented on a layer on top of the original drawing,digital document. Using the display/viewer 240 of the present invention,the user can trace out a path on the 2D window 244 using the basicdrawing tools and have the same path rendered precisely andprogrammatically in the 3D window 242. The path shown in the 3D windowis contextually correct being true to scale and reflecting the accuratemeasurements of a structure. A route virtually walked in the 3D window242 can be visually recorded as a user generated animation, saved andplayed back, as discussed below.

Find Shortest Path Tool

Referring to FIG. 18 A, the Find Shortest Path Tool 300 uses analgorithm to automatically calculate and allow the visual plotting ofthe shortest path between points selected in either the 3D or 2Dwindows. This Find Shortest Path Tool works by transforming the mapmatrix and distance/area data present in the digital document floor planimage and automatically calculates the shortest path. The shortestcalculated path and the associated path true scale measurements areknown and able to be calculated as a result of the scale that isembedded into the header of the TIFF image.

In practice, a user selects the Find Shortest Path tool 300 thengraphically marks start and end points on one digital image in aselected window. For example, an emergency personnel can mark the startpoint 302 as being a doorway in a building and mark the end point 304 asbeing a computer classroom in which a victim or hostage is said to belocated and using the Find Shortest Path tool 300 calculate the shortestpath 306 between the two points. See FIGS. 18A and 18B. The tool 300 canbe used on a single floor, as shown in FIGS. 18A and 18B, or acrossmultiple floors, see Find Shortest Path tool 301 as shown in FIG. 18C.When the tool 301 is used across multiple floors, it will identify andtraverse the buildings unique set of emergency stairwells; entrances andexits during route calculations. See FIG. 18C showing start point 312,end point 314 and route 316.

This tool 300, 301 can assist emergency response or other personnel innavigating from the doorway to the stairwell even when the floor of abuilding is visually or physically obstructed. This Find Shortest Pathtool may also be used by emergency response personnel to determine thefastest route to an emergency location inside a building even beforearriving on scene or when having no prior knowledge of the building'sinterior structure, saving precious time.

In a preferred embodiment, the shortest path is represented in the 2Dwindow 244 by a colored line 306 that follows the exact calculated paththrough the structural drawing and in the 3D window 242 by a red 3Dfloating poly line shape 308. See FIG. 18B. A visual display of the truescale measured path is also shown. After the path is calculated andvisually plotted in both 3D and 2D windows 244, 242 the user canposition the 3D window camera to the preferred structural entry pointand easily visualize, and identify the beginning of the shortest path.The shortest path marker and path directions can be communicated to thefire rescue team or obtained by the team themselves on site.Additionally the calculated route can be automatically played back as awalkthrough animation allowing emergency personnel to see the entireroute and all the building structural elements, objects of interest andhazards along the way.

Path Barrier Tool

FIG. 19 shows the Path Barrier Tool 360, which is used to graphicallymark pathway obstructions and hazards that will prevent passage througha particular area or as a general hazard marker. The user selects theBarrier tool then graphically marks the affected area in either windowto trigger the programmatic placement of a scaled visual “Do Not Enter”marker into the window data sets. This action accurately changes thevisual geometry of the 3D and 2D scenes, 364, 362 respectively, andmathematically alters the makeup of the map matrix used by the Find PathTool.

After the obstructed areas have been marked on at least one of theimages or renderings, the invention can automatically or manuallyrecalculate and visually render the shortest path options taking fullconsideration of the reduced route options. This tool 360 can be used byemergency response personnel to instantly and accurately update tacticalresponse plans and instructions using real time events and userinteraction with the invention to transform map and measurement datainto site specific real time contextual intelligence.

Door Detection Tool

The Door Detection Tool 330, for instance, may be employed with the 3Dwindow 242 and is used to automatically and programmatically count thedoors in a selected path or virtually passed by the invention user. SeeFIG. 20. A running count may be visually displayed to the user of thistool in the 3D window. For instance, a dialogue box may appear to showeach the left door count 332 and right door count 334. The total doorcount can be communicated to the fire rescue team or obtained by theteam themselves on site. Fire rescue teams can use the Door DetectionTool to count the number doors or determine locations of doors in aselected path and even verify that they are on the right path. The DoorDetection Tool is therefore also useful in mitigating navigation throughlow visibility environments.

3D Record Path Tool

The display/viewer 240 may also employ a 3D Record Path Tool 350. SeeFIG. 21. This tool 350 is best used with the 3D window 242, however withsome modifications this tool could also be used in the 2D window 244.

The 3D Record Path Tool 350 allows application users to visually recordevery virtual movement and scene being viewed in the 3D window 242. Whenthe 3D Record Path tool is turned on, the 3D camera's true scale startpoint 352, the exact path taken 354 and current position 356 in the 3Ddigital image are simultaneously and in real-time graphically displayedon the 2D digital image floor plan shown in the 2D window 244. The 3Dwindow 242 will show the 3D view of the path 354 shown in the 2D window.Reference number 358 shows the 3D view taken at the current position 356of the path 354 shown in the 2D window 244. So as the user virtuallywalks through the 3D floor plan model, the user's virtual position isdirectly associated to an exact coordinate on the original 2D floorplan. Also, the exact, true scale measurement of the path's length canbe shown in the measurement box 400.

The visual marker used in the 2D window 244 may be a red line that isdrawn in a visual layer on top of the 2D image. The red line willessentially trace the path the user is walking in the 3D view and plotmarker points or route nodes along the way. A measurement dialog may beemployed with the present invention to display the accurate measuredlength of the recorded path in real time using the embedded scale data.

Once the 3D Record Path tool is stopped, the user not only sees theexact path taken along with starting and ending points, but can replaythe virtual 3D path that was walked in the 3D window 242 or save theentire 3D visual sequence, i.e. animated movie, out to a standard .avi,mp4 or a variety of other standard video formats for distribution.

The 3D Record Path functionality can be used in a variety of situationsand contexts. For example, a user can create an accurate, to-scalevirtual model of a museum floor plan by scanning paper drawings of asection of a museum. When the 3D digital image is created, anyfurniture, sculpture, and wall hanging art objects can be accuratelymodeled and represented inside the 3D museum floor plan. The applicationuser could then walk through the virtual 3D museum and initiate the 3DRecord Path tool to view an accurate visual representation (or movie) ofthe path taken. This may be useful to guide museum patrons through amuseum exhibit, or allow a remote museum curator to view exact accuratemuseum and art layouts in another country without physically beingpresent at the viewed museum. This tool could also be used to marketmaterials and virtual art shows as well as to curate entire shows fromremote locations without the expense of onsite visits.

In another instance, the 3D Record Path Tool could be employed byemergency crews to gain invaluable insight into the interior layouts ofbuildings during emergency situations. Using the 3D Record Path Tool,tactical information gathered on scene can be used in conjunction withthe 3D floor plan models and allow users to gain a detailed, fullyto-scale visual guide of emergency locations. Also, with this tool, theuser can plot entry or exit routes in a structure, automatically viewthe entire length of the path in a true scale 3D animation andintelligently adjust the navigation path based on architectural andhazard elements that can only be visualized and incorporated into thedigital images using the present invention. These recorded 3D floor planvisualizations can be used to make further tactical decisions based onthe information being rendered and the novel manner of presentation atcommand and control or sent out into the field as pre-planning guidesfor emergency responders on scene. All these uses maintain accuratescale and measurement data throughout because of the embedded scale inthe TIFF header. It should also be noted that the raster image createdusing the tools of the present invention is transmitted from one sourceto another maintaining true scale throughout the entire process and enduser activity.

Evacuation Simulation Tool

The Evacuation Simulation Tool allows users of the 3D/2D synchronizedviewer 240 to extend the data calculated by the shortest path tool 300to larger scale simulations. In essence, once a shortest path has beencalculated by the Shortest Path Tool 300 and the path has beengraphically displayed in the 3D and 2D viewports, the end user canselect the path in the 3D or 2D window and activate a new functionthrough the right click menu.

This Evacuation Simulation tool would work with Population Density dataencoded in the building 3D/2D data set to describe the number of personsthat are in the vicinity of the Shortest Path Start Point. TheEvacuation Simulation tool would then use the true scale dimensioninformation embedded in the tiff header to calculate the dimension ofthe surrounding area and the area taken by the estimated population. Incombination with a virtual crowd simulation, point particle method orflock algorithm the Evacuation Simulation tool would then calculate howlong it will take to move the total estimated number of people encodedin the Population Density data along the true scale calculated shortestpath, through all the structural bottlenecks to the end path location.Users would simulate/select either a time of day or an incidentdescription which would alter the estimated Population Density data inthe building 3D and 2D data sets. Then the simulator would take intoaccount the prepared density information for the various parts of thebuilding that the path traverses. Time estimates would be displayed backto the end use as well as the ability to run a visual playback of thegroup of virtual people moving along the evacuation route.

Each of the tools described above can be used alone or in anycombination. For instance, the user can combine the Find Shortest PathTool 300, 301 and the 3D Record Path Tool 360 to not only follow theshortest path in one window that is simultaneously being plotted in thecorresponding window, but also view a recorded movie of the selectedpath taken from the 3D digital rendering.

Using the present invention, the user may also manually record on eitherdigital image the presence of additional hazards, obstructions or otherdata noticed on-scene or while watching the replay of the recorded 3Dvirtual path from the 3D Record Path Tool. This added information can bethen distributed/relayed to the other emergency responders inside orwithin the real world building location or else where permitting quickentry and exit of the building. This feature allows emergency teamleaders and command and control personnel to continually update theirresponse plans as new information is obtained by using the invention.Information that, would in part, not be available to the end userwithout the invention.

Tracking Real World Emergency Responder

The true scale, coordinate-matched, linked in-real time, dualthree-dimensional/two-dimensional visual display/viewer 240 can be usedto track real world objects (assets) and simultaneously mark,graphically, their location on a 2D floor plan and as a correspondingobject in the 3D window 242. This combination of contextually relatedvisual environments enables the transformation of two independentstreams of data into one coherent piece of information.

In practice an emergency responder can be outfitted with atransmitter/receiver device. These devices use a location method toidentify where an asset is in relation to some scaled, coordinate basedsystem. For example, as mentioned above, asset coordinates can beobtained in relation to the GPS Geographic Coordinate system or alocalized location grid that creates a grid entity and ties assetlocation to a coordinate within that localized grid.

For example, a responder can be outfitted with a GPS device capable ofreceiving/transmitting location or identification signals to either areceiver location outside of a building or a receiver already existingwithin the structure which may be outfitted with a disaster proof signalrepeating and amplifying unit. These repeating signals and amplifyingunits are designed to pick up the lower strength/lower powered personalGPS device signals and retransmit the signals to external receivers sothat location data can be calculated.

In another example, the responder can also be outfitted with a radiotransmitter and receiver, this can be referred to as the asset tag. Insystems that are well known in the art, a localized set of computerconnected mobile transceivers send out radio signals and receive signalsback from the asset tags. The present invention may employ atriangulation or a multilateration method, known in the art, indetecting coordinates for an asset or emergency responder. Depending onthe type of system implemented the asset tags location can be determinedin reference to this localized grid in a variety of ways depending onthe type, strength and broadcast reach of the asset tag and receiverstations.

Once an object's location (such as an emergency responder's location) iscalculated using the methods described above the coordinate grid andscale can be translated to align with the coordinate system of thepresent invention. A reference point common among all the coordinatesystems is established so that locations can be plotted in relation tothis point.

The asset coordinate data of the emergency responder can be graphicallyrendered simultaneously using the 3D/2D dual visual display/viewer 240of the present invention. As the asset location coordinate data isupdated, the location of the object or emergency responder is alsoupdated in the 3D and 2D digital images.

The display/viewer 240 and the tools of the present invention, providethe user with an understanding of the contextual 3D/2D visualenvironment by allowing the user to see an object's (or asset's)location rendered in a manner not realized by the prior art. The presentinvention is also useful as an analysis tool. For instance, if an assetis navigating a structure in a compromised, hazardous, or visuallyimpaired situation the immediate dangers, percentage of route completionand alternative routes in reaction to changing conditions on scene canbe evaluated and reacted to with information transformed by the presentinvention.

EXAMPLE

The present invention may be described using the below non-limitingexample.

A fire is reported burning on the third floor of a four-story structure.The emergency responders uses a laptop 28 a to access the floor plandata of the burning building either on-site or on route to the building.Once the floor is located, the emergency responder can open the desireddigital images using the display/viewer 240 and manipulate the data inseveral different ways all of which can occur simultaneously.

The user can identify all entry and exit points for the structure usingthe 2D window 244 in the digital image display/viewer 240 and the selectthe Find Shortest Path Tool to determine the shortest and fastest pathfrom one point on the floor to an exit point. Once the path has beencalculated and visually plotted in both the 3D and 2D windows 242, 244the user can place the 3D window camera to the preferred structuralentry point or their current position in the building and view thecalculated path. The 3D Record Path tool may be employed at this time torecord the path taken in the 3D digital image (3D virtual model) of thebuilding. This video can be replayed back or distributed by wire orwirelessly to other emergency personnel. Native to the standard videoformats created, end users of this type of system transformed, usergenerated content will not need to be system users themselves to sharein the visual intelligence.

If real-time, on-scene date being received by emergency crews indicatesthat the path shown by the Find Shortest Path tool is obstructed orcannot be taken, then the user can reroute the path using the FindShortest Path tool to calculate another path to the exit point. Usingthe counterpart Barrier Tool 360 users can mark the hazard/obstructionidentified by the on scene emergency crew in the 3D or 2D windowtriggering the simultaneous insertion of a visual to-scalerepresentations of the obstruction. This visual representation can be ofthe actual obstruction or a geometric shape textured mapped with aglobally standard emergency symbol. Users can mark multiple to scalebarriers altering the data parameters of the mapped environment andultimately affecting the intelligence and measured route optionscalculated from the underlying document image data. Finally, to assistthe emergency personnel in way finding the smoke-filled floor, the DoorDetection Tool may be used to count the number of doorways or entrywaysuntil the desired end point is reached. This data is essential tovisually impaired environments where tactile verification is needed fornavigation. This data can be found on almost any typical architecturalfloor plan but it must be manually tabulated along a desired route forthe data to be meaningful or offer any usable intelligence. The use ofthe current invention overcomes this serious deficiency in the art.

If the building is internally equipped with a receiver/amplifier and theemergency responder involved in an interior attack or as part of arescue company in a fire hot zone is equipped with a GPS device or radiotransmitter, outside personnel may be able to visually track theresponder's exact coordinate within the third floor of the building andverbally instruct the responder to any new occurrences or changes to theemergency situation or with additional information based on the systemsvisual display to assist the responder in dealing with conditions thatthey are discovering and reporting back to outside personnel in theinterior of the building.

Many modifications and other embodiments of the inventions set forthherein will come to mind to one skilled in the art to which theseinventions pertain having the benefit of the teachings presented in theforegoing descriptions and the associated drawings. Therefore, it is tobe understood that the inventions are not to be limited to the specificembodiments disclosed and that modifications and other embodiments areintended to be included within the scope of the appended claims.Although specific terms are employed herein, they are used in a genericand descriptive sense only and not for purposes of limitation.

1. A method of tracking assets within a structure using a digital viewerin which the digital viewer includes a two dimensional display meansdisplaying a two dimensional digital image of a scanned paper documentof a structure, scale information of the paper document being embeddedin a header of the two dimensional digital image, wherein theimprovement comprises: displaying a three dimensional rendering of saidtwo dimensional digital image in a three dimensional display means, saidthree dimensional rendering having a three dimensional coordinatesystem, said three dimensional coordinate system being a non-renderedgrid disposed on said three dimensional rendering, said two dimensionaldigital image having a two dimensional coordinate system, said twodimensional coordinate system being a non-rendered grid disposed on saidtwo dimensional digital image; providing a common coordinate system,wherein said common coordinate system is formed by matching saidnon-rendered grid of said two dimensional digital image to saidnon-rendered grid of said three dimensional rendering; providing areceiver location about the structure; and providing an asset with areceiving/transmitting device; wherein location of said asset withinsaid structure can be detected on said common coordinate system, andwherein said two dimensional digital image is linked in real-time andcoordinate matched with said three dimensional rendering.
 2. The methodof claim 1, wherein coordinates of said location of said asset isdetermined using a triangulation method on said digital viewer.
 3. Themethod of claim 1, wherein coordinates of said location of said asset isdetermined using a multilateration method on said digital viewer.
 4. Themethod of claim 1, wherein the receiving/transmitting device is a radiotransmitter and receiver device.
 5. The method of claim 1, wherein thereceiving/transmitting device is a GPS device.
 6. The method of claim 5,wherein the GPS device has a GPS coordinate system and a GPS scale, saidGPS coordinate system and said GPS scale being translated to align withthe common coordinate system.
 7. The method of claim 1, furthercomprising a common referenced point, the grid of the 2D digital imageand the grid of the 3D rendering being linked to the common referencedpoint.
 8. The method of claim 7, wherein the common referenced point isa corner of a plan.
 9. The method of claim 7, wherein the commonreferenced point is an upper-left corner of a plan serving as a 0,0point.